Parallel Training of Neural Networks for Speech Recognition

Tato diplomová práce je zaměřena na paralelizaci trénování neuronových sítí pro rozpoznávání řeči. V rámci této diplomové práce byly implementovány a porovnány dvě strategie paralelizace. První strategií je paralelizace dat s využitím rozdělení trénování do několika POSIX vláken. Druhou strategií je paralelizace uzlů s využitím platformy pro obecné výpočty na grafických kartách CUDA. V případě první strategie bylo dosaženo 4x urychlení, v případě využití platformy CUDA bylo dosaženo téměř 10x urychlení. Pro trénování byl použit algoritmus Stochastic Gradient Descent se zpětným šířením chyb. Po krátkém úvodu následuje druhá kapitola práce, která je motivační a zasazuje probém do kontextu rozpoznávání řeči. Třetí kapitola práce je teoretická a diskutuje neuronové sítě a metodu trénování. Následující kapitoly jsou zaměřené na návrh a implementaci a popisují iterativní vývoj tohoto projektu. Poslední obsáhlá kapitola popisuje testovací systém a uvádí výsledky provedených experimentů. V závěru jsou krátce zhodnoceny dosažené výsledky a nastíněna perspektiva dalšího vývoje projektu.This thesis deals with different parallelizations of training procedure for artificial neural networks. The networks are trained as phoneme-state acoustic descriptors for speech recognition. Two effective parallelization strategies were implemented and compared. The first strategy is data parallelization, where the training is split into several POSIX threads. The second strategy is node parallelization, which uses CUDA framework for general purpose computing on modern graphic cards. The first strategy showed a 4x speed-up, while using the second strategy we observed nearly 10x speed-up. The Stochastic Gradient Descent algorithm with error backpropagation was used for the training. After a short introduction, the second chapter of this thesis shows the motivation and introduces the neural networks into the context of speech recognition. The third chapter is theoretical, the anatomy of a neural network and the used training method are discussed. The following chapters are focused on the design and implementation of the project, while the phases of the iterative development are described. The last extensive chapter describes the setup of the testing system and reports the experimental results. Finally, the obtained results are concluded and the possible extensions of the project are proposed.

Overview of Selected Issues Related to Soldering

The formation of defects and imperfections in the soldering process can have many causes, which primarily include a poorly setup technological process, inappropriate or inappropriately used materials and their combinations, the effect of the surroundings and design errors. This chapter lists some examples of errors that can occur in soldering, while review is devoted to selected defects: non-wettability of the solder pads, dewetting, wrong solder mask design, warpage, head-in-pillow, cracks in the joints, pad cratering, black pad, solder beading, tombstoning, dendrites, voids, flux spattering from the solder paste, popcorning and whiskers

BUT CHiME-7 system description

This paper describes the joint effort of Brno University of Technology (BUT), AGH University of Krakow and University of Buenos Aires on the development of Automatic Speech Recognition systems for the CHiME-7 Challenge. We train and evaluate various end-to-end models with several toolkits. We heavily relied on Guided Source Separation (GSS) to convert multi-channel audio to single channel. The ASR is leveraging speech representations from models pre-trained by self-supervised learning, and we do a fusion of several ASR systems. In addition, we modified external data from the LibriSpeech corpus to become a close domain and added it to the training. Our efforts were focused on the far-field acoustic robustness sub-track of Task 1 - Distant Automatic Speech Recognition (DASR), our systems use oracle segmentation.Comment: 6 pages, Chime-7 challenge 202

Pasivní motodlaha v traumatologii a ortopedii

This article deals with analysis of the knee continuous passive motion (CPM) splint used for rehabilitation of patients after surgeries and injuries. CPM splints are used for speeding up the treatment and restoration of the joint mobility and to stop complications caused by immobilization. The goal is to solve kinematic, dynamic, deformation and stress parameters of “Artromot K1 Comfort” CPM splint.Tento článek se zabývá analýzou pasivní kolenní motodlahy sloužící k rehabilitaci pacientů po operacích a úrazech. Pasivní motodlahy se používají pro urychlení léčby, obnovu pohyblivosti kloubů a zamezení komplikacím způsobených imobilizací. Cílem je řešení kinematických, dynamických, deformačních a pevnostních parametrů pro motodlahu “Artromot K1 Comfort”

Flavonoid glycosides from endemic bulgarian astragalus aitosensis (Ivanisch.)

© 2019 The Author(s). Background: The activity and haemolytic toxicity associated with primaquine has been linked to its reactive metabolites. The reactive metabolites are thought to be primarily formed through the action of cytochrome P 450 -mediated pathways. Human erythrocytes generally are not considered a significant contributor to drug biotransformation. As erythrocytes are the target of primaquine toxicity, the ability of erythrocytes to mediate the formation of reactive oxidative primaquine metabolites in the absence of hepatic enzymes, was evaluated. Methods: Primaquine and its enantiomers were incubated separately with human red blood cells and haemoglobin. Post-incubation analysis was performed with UPLC-MS/MS to identify products of biotransformation. Results: The major metabolite detected was identified as primaquine-5,6-orthoquinone, reflecting the pathway yielding putative active and haematotoxic metabolites of primaquine, which was formed by oxidative demethylation of 5-hydroxyprimaquine. Incubation of primaquine with haemoglobin in a cell-free system yielded similar results. It appears that the observed biotransformation is due to non-enzymatic processes, perhaps due to reactive oxygen species (ROS) present in erythrocytes or in the haemoglobin incubates. Conclusion: This study presents new evidence that primaquine-5,6-orthoquinone, the metabolite of primaquine reflecting the oxidative biotransformation pathway, is generated in erythrocytes, probably by non-enzymatic means, and may not require transport from the liver or other tissues

Patrol team language identification system for DARPA RATS P1 evaluation

This paper describes the language identification (LID) system developed by the Patrol team for the first phase of the DARPA RATS (Robust Automatic Transcription of Speech) program, which seeks to advance state of the art detection capabilities on audio from highly degraded communication channels. We show that techniques originally developed for LID on telephone speech (e.g., for the NIST language recognition evaluations) remain effective on the noisy RATS data, provided that careful consideration is applied when designing the training and development sets. In addition, we show significant improvements from the use of Wiener filtering, neural network based and language dependent i-vector modeling, and fusion

IMRT using simultaneous integrated boost (66 Gy in 6 weeks) with and without concurrent chemotherapy in head and neck cancer – toxicity evaluation

AimTo evaluate the toxicity of intensity-modulated radiotherapy with simultaneous integrated boost (SIB-IMRT) in head and neck cancer patients treated using a protocol comprising 66 Gy to the PTV1 (planning target volume; region of macroscopic tumour) and 60 Gy and 54 Gy to the regions with high risk (PTV2) and low risk (PTV3) of subclinical disease in 30 fractions in six weeks.Material and MethodsBetween December 2003 and February 2006, 48 patients (median age 55; range 25–83, performance status 0–1) with evaluable non-metastatic head and neck cancer of various localizations and stages (stages: I–1; II–8; III–12; IV–27 patients, resp.) were irradiated according to the protocol and followed (median follow-up 20 months; range 4–42). Ten patients underwent concurrent chemotherapy (CT) and in 15 patients the regimen was indicated postoperatively because of close or positive margins. In all cases the regimen was used as an alternative to conventional radiotherapy (70 Gy in 7 weeks). The acute and late toxicities were evaluated according to RTOG and RTOG/EORTC toxicity scales, respectively.ResultsAll patients finished the treatment without the need for interruption due to acute toxicity. No patient experienced grade 4 toxicity. More severe acute toxicity was observed in patients with CT, but the most severe toxicity was grade 3. Grade 3 toxicity was observed in the skin, mucous membrane, salivary glands, pharynx/oesophagus and larynx in 8.4%, 35.4%, 39.6% and 2.1%, in the CT subgroup in 10%, 100%, 90%, 10%, respectively. The trend of impairment of acute toxicity by concurrent chemotherapy was statistically confirmed by Fisher's exact test (for mucous membranes p=0.000002 and pharyngeal/oesophageal toxicity p=0.0004). The most severe late toxicity was grade 2 subcutaneous tissue (34.2%), mucous membrane (36.8%) and larynx (11.1%), grade 3 in salivary gland (2.6%) and grade 1 in skin (84.2%) and spinal cord (5.4%). The late toxicity was not increased by chemotherapy.ConclusionIn light of the toxicity profile we consider the presented regimen to be an alternative to conventional radiotherapy 70 Gy in 7 weeks. The addition of CT requires more intensive supportive care

ATCO2 corpus: A Large-Scale Dataset for Research on Automatic Speech Recognition and Natural Language Understanding of Air Traffic Control Communications

Personal assistants, automatic speech recognizers and dialogue understanding systems are becoming more critical in our interconnected digital world. A clear example is air traffic control (ATC) communications. ATC aims at guiding aircraft and controlling the airspace in a safe and optimal manner. These voice-based dialogues are carried between an air traffic controller (ATCO) and pilots via very-high frequency radio channels. In order to incorporate these novel technologies into ATC (low-resource domain), large-scale annotated datasets are required to develop the data-driven AI systems. Two examples are automatic speech recognition (ASR) and natural language understanding (NLU). In this paper, we introduce the ATCO2 corpus, a dataset that aims at fostering research on the challenging ATC field, which has lagged behind due to lack of annotated data. The ATCO2 corpus covers 1) data collection and pre-processing, 2) pseudo-annotations of speech data, and 3) extraction of ATC-related named entities. The ATCO2 corpus is split into three subsets. 1) ATCO2-test-set corpus contains 4 hours of ATC speech with manual transcripts and a subset with gold annotations for named-entity recognition (callsign, command, value). 2) The ATCO2-PL-set corpus consists of 5281 hours of unlabeled ATC data enriched with automatic transcripts from an in-domain speech recognizer, contextual information, speaker turn information, signal-to-noise ratio estimate and English language detection score per sample. Both available for purchase through ELDA at http://catalog.elra.info/en-us/repository/browse/ELRA-S0484. 3) The ATCO2-test-set-1h corpus is a one-hour subset from the original test set corpus, that we are offering for free at https://www.atco2.org/data. We expect the ATCO2 corpus will foster research on robust ASR and NLU not only in the field of ATC communications but also in the general research community.Comment: Manuscript under review; The code will be available at https://github.com/idiap/atco2-corpu